ReadOnlySequence<T>

ReadOnlySequence in .NET

Master efficient buffer management in .NET with ReadOnlySequence<T> using free flashcards and spaced repetition practice. This lesson covers discontiguous memory representation, segment navigation, and performance optimization—essential concepts for building high-performance .NET applications that minimize allocations and copies.

Welcome to ReadOnlySequence 💻

When working with streaming data, network protocols, or large file processing in .NET, you'll often encounter data that doesn't arrive in a single contiguous block. Traditional arrays and Span<T> work beautifully for contiguous memory, but what happens when your data is scattered across multiple buffers? This is where ReadOnlySequence<T> becomes indispensable.

ReadOnlySequence<T> is a struct in the System.Buffers namespace that represents a read-only sequence of items that may span multiple memory segments. Think of it as a linked list of memory segments that you can traverse efficiently without copying data. It's the foundation of high-performance I/O in modern .NET, powering System.IO.Pipelines, Kestrel web server, and many other performance-critical components.

Why ReadOnlySequence Exists 🎯

Imagine receiving data over a network. The data arrives in chunks:

• First packet: 512 bytes
• Second packet: 1024 bytes
• Third packet: 256 bytes

With traditional approaches, you'd either:

1. Copy everything into a single large buffer (expensive, causes allocations)
2. Process each chunk separately (complex logic, potential protocol issues)

ReadOnlySequence<T> provides a third option: treat the disconnected chunks as a single logical sequence without copying. You get unified sequential access while the underlying memory remains fragmented.

Traditional Approach (Copying):
┌─────────┐  ┌─────────┐  ┌─────────┐
│Chunk 1  │  │Chunk 2  │  │Chunk 3  │
└────┬────┘  └────┬────┘  └────┬────┘
     │           │           │
     └───────────┴───────────┘
                 ▼
     ┌───────────────────────────┐
     │  Large Copied Buffer      │ ❌ Allocation!
     └───────────────────────────┘

ReadOnlySequence Approach (Zero-Copy):
┌─────────┐  ┌─────────┐  ┌─────────┐
│Chunk 1  │→ │Chunk 2  │→ │Chunk 3  │
└─────────┘  └─────────┘  └─────────┘
      ↑                              ↑
      └──── ReadOnlySequence ────────┘
            ✅ No copying, unified view!

Core Concepts 🧠

1. Structure and Representation

ReadOnlySequence<T> is a struct (value type) that wraps either:

• Single segment: Backed by a single ReadOnlyMemory<T> (contiguous)
• Multiple segments: Backed by a linked sequence of memory blocks

The beauty is that you interact with it the same way regardless of whether it's single or multi-segment.

Property	Type	Description
Length	long	Total number of items across all segments
IsEmpty	bool	True if sequence contains zero items
IsSingleSegment	bool	True if backed by single contiguous memory
First	ReadOnlyMemory<T>	First segment's memory
Start	SequencePosition	Position pointing to first item
End	SequencePosition	Position pointing past last item

2. SequencePosition: The Navigation Token 🧭

SequencePosition is an opaque struct that represents a position within a ReadOnlySequence<T>. Think of it as a bookmark—it doesn't contain the data itself, but points to a specific location.

public readonly struct SequencePosition
{
    // Internal: tracks which segment and offset within that segment
    // You rarely construct these manually—methods return them
}

Key characteristics:

• Opaque: You can't inspect its internals directly
• Relative: Only valid for the sequence it came from
• Lightweight: Just two fields internally (object reference + integer index)

💡 Tip: Treat SequencePosition like an iterator in C++—it's meaningless outside its original sequence.

3. Creating ReadOnlySequence Instances

From Single Segment (Contiguous Memory)

byte[] array = new byte[1024];
ReadOnlyMemory<byte> memory = array;
var sequence = new ReadOnlySequence<byte>(memory);

Console.WriteLine($"IsSingleSegment: {sequence.IsSingleSegment}"); // True
Console.WriteLine($"Length: {sequence.Length}"); // 1024

From Multiple Segments (Custom Implementation)

For multi-segment sequences, you typically implement ReadOnlySequenceSegment<T>. This is an abstract class that forms a linked list:

public class BufferSegment : ReadOnlySequenceSegment<byte>
{
    public BufferSegment(ReadOnlyMemory<byte> memory)
    {
        Memory = memory;
    }

    public BufferSegment Append(ReadOnlyMemory<byte> memory)
    {
        var segment = new BufferSegment(memory)
        {
            RunningIndex = RunningIndex + Memory.Length
        };
        Next = segment;
        return segment;
    }
}

Creating a multi-segment sequence:

// Create three separate buffers
var buffer1 = new byte[512];
var buffer2 = new byte[1024];
var buffer3 = new byte[256];

// Build linked segment chain
var firstSegment = new BufferSegment(buffer1);
var secondSegment = firstSegment.Append(buffer2);
var thirdSegment = secondSegment.Append(buffer3);

// Create sequence from first and last segments
var sequence = new ReadOnlySequence<byte>(
    firstSegment, 0,           // Start: first segment, offset 0
    thirdSegment, buffer3.Length  // End: last segment, its length
);

Console.WriteLine($"IsSingleSegment: {sequence.IsSingleSegment}"); // False
Console.WriteLine($"Length: {sequence.Length}"); // 1792 (512+1024+256)

Linked Segment Structure:
┌──────────────────────────────────┐
│ BufferSegment 1                  │
│ Memory: byte[512]                │
│ RunningIndex: 0                  │
│ Next: ───────────────────┐       │
└──────────────────────────┼───────┘
                           ▼
┌──────────────────────────────────┐
│ BufferSegment 2                  │
│ Memory: byte[1024]               │
│ RunningIndex: 512                │
│ Next: ───────────────────┐       │
└──────────────────────────┼───────┘
                           ▼
┌──────────────────────────────────┐
│ BufferSegment 3                  │
│ Memory: byte[256]                │
│ RunningIndex: 1536               │
│ Next: null                       │
└──────────────────────────────────┘

4. Traversing the Sequence 🚶‍♂️

Fast Path: Single Segment

Always check IsSingleSegment first for optimal performance:

public void ProcessSequence(ReadOnlySequence<byte> sequence)
{
    if (sequence.IsSingleSegment)
    {
        // Fast path: direct span access
        ReadOnlySpan<byte> span = sequence.First.Span;
        ProcessSpan(span);
    }
    else
    {
        // Slow path: iterate segments
        ProcessMultiSegment(sequence);
    }
}

Iterating Through Segments

Use ReadOnlySequence<T>.GetEnumerator() to walk through segments:

public int CountBytes(ReadOnlySequence<byte> sequence)
{
    int total = 0;
    
    foreach (ReadOnlyMemory<byte> segment in sequence)
    {
        total += segment.Length;
        Console.WriteLine($"Segment: {segment.Length} bytes");
    }
    
    return total;
}

Manual iteration with positions:

public void ProcessWithPositions(ReadOnlySequence<byte> sequence)
{
    SequencePosition position = sequence.Start;
    
    while (sequence.TryGet(ref position, out ReadOnlyMemory<byte> memory))
    {
        ReadOnlySpan<byte> span = memory.Span;
        // Process this segment
        ProcessSegment(span);
    }
}

5. Slicing Operations ✂️

One of the most powerful features is zero-copy slicing:

ReadOnlySequence<byte> sequence = GetDataFromNetwork();

// Get first 100 bytes
ReadOnlySequence<byte> header = sequence.Slice(0, 100);

// Get everything after position 100
ReadOnlySequence<byte> body = sequence.Slice(100);

// Slice between two positions
SequencePosition start = sequence.GetPosition(50);
SequencePosition end = sequence.GetPosition(150);
ReadOnlySequence<byte> middle = sequence.Slice(start, end);

⚠️ Important: Slicing creates a new ReadOnlySequence<T> that references the same underlying memory—no copying occurs!

6. Searching Within Sequences 🔍

Common pattern: finding delimiters (like newlines in text protocols):

public ReadOnlySequence<byte> ReadLine(ReadOnlySequence<byte> buffer)
{
    SequencePosition? position = buffer.PositionOf((byte)'\n');
    
    if (position == null)
    {
        return default; // No newline found
    }
    
    // Return everything up to (but not including) the newline
    return buffer.Slice(0, position.Value);
}

Searching for multi-byte patterns:

public SequencePosition? FindPattern(
    ReadOnlySequence<byte> sequence, 
    ReadOnlySpan<byte> pattern)
{
    var reader = new SequenceReader<byte>(sequence);
    
    while (!reader.End)
    {
        if (reader.IsNext(pattern, advancePast: false))
        {
            return reader.Position;
        }
        reader.Advance(1);
    }
    
    return null;
}

Practical Examples 💼

Example 1: HTTP Protocol Parser

public class HttpRequestParser
{
    private static readonly byte[] NewLine = new byte[] { (byte)'\r', (byte)'\n' };
    private static readonly byte[] DoubleNewLine = 
        new byte[] { (byte)'\r', (byte)'\n', (byte)'\r', (byte)'\n' };

    public bool TryParseRequest(
        ReadOnlySequence<byte> buffer,
        out HttpRequest request,
        out SequencePosition consumed)
    {
        request = null;
        consumed = buffer.Start;

        // Find end of headers (double CRLF)
        SequencePosition? headersEnd = buffer.PositionOf(DoubleNewLine);
        if (headersEnd == null)
        {
            return false; // Incomplete request
        }

        // Extract request line and headers
        ReadOnlySequence<byte> headersSequence = 
            buffer.Slice(0, headersEnd.Value);

        // Parse request line (first line)
        SequencePosition? firstLineEnd = headersSequence.PositionOf(NewLine);
        if (firstLineEnd == null) return false;

        ReadOnlySequence<byte> requestLine = 
            headersSequence.Slice(0, firstLineEnd.Value);
        
        // Convert to string for parsing (necessary here)
        string requestLineStr = Encoding.UTF8.GetString(requestLine);
        var parts = requestLineStr.Split(' ');

        request = new HttpRequest
        {
            Method = parts[0],
            Path = parts[1],
            Version = parts[2]
        };

        // Mark how much we consumed
        consumed = buffer.GetPosition(4, headersEnd.Value); // Skip \r\n\r\n
        return true;
    }
}

Why this is efficient:

• No copying of buffer data until absolutely necessary (encoding to string)
• Works regardless of how network packets arrived
• Can process partial requests incrementally

Example 2: Length-Prefixed Message Decoder

public class MessageDecoder
{
    // Protocol: [4-byte length][message bytes]
    public bool TryDecodeMessage(
        ReadOnlySequence<byte> buffer,
        out ReadOnlySequence<byte> message,
        out SequencePosition consumed)
    {
        message = default;
        consumed = buffer.Start;

        // Need at least 4 bytes for length prefix
        if (buffer.Length < 4)
        {
            return false;
        }

        // Read length (handles cross-segment reading)
        int messageLength = ReadInt32(buffer.Slice(0, 4));

        // Check if full message is available
        if (buffer.Length < 4 + messageLength)
        {
            return false;
        }

        // Extract message (zero-copy slice)
        message = buffer.Slice(4, messageLength);
        
        // Mark position after this message
        consumed = buffer.GetPosition(4 + messageLength);

        return true;
    }

    private int ReadInt32(ReadOnlySequence<byte> sequence)
    {
        // Fast path: single segment
        if (sequence.IsSingleSegment)
        {
            return BinaryPrimitives.ReadInt32BigEndian(sequence.First.Span);
        }

        // Slow path: copy to stack buffer
        Span<byte> buffer = stackalloc byte[4];
        sequence.CopyTo(buffer);
        return BinaryPrimitives.ReadInt32BigEndian(buffer);
    }
}

Key patterns demonstrated:

• Length checking before attempting to read
• Efficient integer reading with fast/slow paths
• Position tracking for consuming data
• stackalloc for temporary small buffers

Example 3: JSON Token Scanner

public class JsonTokenizer
{
    public IEnumerable<JsonToken> Tokenize(ReadOnlySequence<byte> json)
    {
        var reader = new SequenceReader<byte>(json);
        
        while (!reader.End)
        {
            reader.AdvancePastAny((byte)' ', (byte)'\t', (byte)'\r', (byte)'\n');
            
            if (reader.End) break;

            if (reader.TryPeek(out byte current))
            {
                switch ((char)current)
                {
                    case '{':
                        reader.Advance(1);
                        yield return new JsonToken(TokenType.OpenBrace);
                        break;
                        
                    case '}':
                        reader.Advance(1);
                        yield return new JsonToken(TokenType.CloseBrace);
                        break;
                        
                    case '"':
                        yield return ReadString(ref reader);
                        break;
                        
                    case '-':
                    case '0': case '1': case '2': case '3': case '4':
                    case '5': case '6': case '7': case '8': case '9':
                        yield return ReadNumber(ref reader);
                        break;
                        
                    default:
                        yield return ReadKeyword(ref reader);
                        break;
                }
            }
        }
    }

    private JsonToken ReadString(ref SequenceReader<byte> reader)
    {
        reader.Advance(1); // Skip opening quote
        SequencePosition start = reader.Position;
        
        // Find closing quote (simplified - doesn't handle escapes)
        while (reader.TryRead(out byte b))
        {
            if (b == '"')
            {
                break;
            }
        }
        
        return new JsonToken(TokenType.String);
    }
}

Why SequenceReader matters here:

• Provides high-level reading operations
• Handles segment boundaries automatically
• Maintains position tracking
• Supports lookahead (TryPeek) without advancing

Example 4: Custom BufferWriter Integration

public class ChunkedBufferWriter : IBufferWriter<byte>
{
    private readonly List<byte[]> _segments = new();
    private byte[] _current;
    private int _position;
    private const int ChunkSize = 4096;

    public ChunkedBufferWriter()
    {
        _current = new byte[ChunkSize];
        _segments.Add(_current);
    }

    public void Advance(int count)
    {
        _position += count;
        
        if (_position >= ChunkSize)
        {
            _current = new byte[ChunkSize];
            _segments.Add(_current);
            _position = 0;
        }
    }

    public Memory<byte> GetMemory(int sizeHint = 0)
    {
        int available = ChunkSize - _position;
        
        if (sizeHint > available)
        {
            _current = new byte[Math.Max(ChunkSize, sizeHint)];
            _segments.Add(_current);
            _position = 0;
        }
        
        return _current.AsMemory(_position);
    }

    public Span<byte> GetSpan(int sizeHint = 0) => 
        GetMemory(sizeHint).Span;

    // Convert written data to ReadOnlySequence
    public ReadOnlySequence<byte> AsSequence()
    {
        if (_segments.Count == 1)
        {
            return new ReadOnlySequence<byte>(
                _segments[0].AsMemory(0, _position));
        }

        var firstSegment = new BufferSegment(_segments[0]);
        var currentSegment = firstSegment;

        for (int i = 1; i < _segments.Count - 1; i++)
        {
            currentSegment = currentSegment.Append(_segments[i]);
        }

        // Last segment uses only written portion
        currentSegment = currentSegment.Append(
            _segments[^1].AsMemory(0, _position));

        return new ReadOnlySequence<byte>(
            firstSegment, 0,
            currentSegment, _position);
    }
}

Usage:

var writer = new ChunkedBufferWriter();

// Write data that spans multiple chunks
for (int i = 0; i < 10000; i++)
{
    var span = writer.GetSpan(4);
    BinaryPrimitives.WriteInt32LittleEndian(span, i);
    writer.Advance(4);
}

// Get as sequence without copying
ReadOnlySequence<byte> sequence = writer.AsSequence();
Console.WriteLine($"Total bytes: {sequence.Length}");
Console.WriteLine($"Segments: {sequence.IsSingleSegment ? 1 : "multiple"}");

Common Mistakes ⚠️

Mistake 1: Copying Data Unnecessarily

❌ Wrong:

public byte[] ProcessSequence(ReadOnlySequence<byte> sequence)
{
    // Defeats the entire purpose!
    byte[] array = sequence.ToArray();
    return ProcessArray(array);
}

✅ Right:

public void ProcessSequence(ReadOnlySequence<byte> sequence)
{
    if (sequence.IsSingleSegment)
    {
        ProcessSpan(sequence.First.Span);
    }
    else
    {
        foreach (ReadOnlyMemory<byte> segment in sequence)
        {
            ProcessSpan(segment.Span);
        }
    }
}

Mistake 2: Assuming Contiguous Memory

❌ Wrong:

public int ReadInt32(ReadOnlySequence<byte> sequence)
{
    // Crashes if sequence spans segments!
    return BinaryPrimitives.ReadInt32LittleEndian(
        sequence.First.Span);
}

✅ Right:

public int ReadInt32(ReadOnlySequence<byte> sequence)
{
    if (sequence.Length < 4)
        throw new ArgumentException("Too short");
    
    if (sequence.IsSingleSegment)
    {
        return BinaryPrimitives.ReadInt32LittleEndian(
            sequence.First.Span);
    }
    
    Span<byte> temp = stackalloc byte[4];
    sequence.Slice(0, 4).CopyTo(temp);
    return BinaryPrimitives.ReadInt32LittleEndian(temp);
}

Mistake 3: Keeping SequencePosition After Sequence Changes

❌ Wrong:

public void ProcessMessages()
{
    var buffer = GetBuffer();
    SequencePosition position = buffer.GetPosition(100);
    
    // Buffer gets modified/recycled
    buffer = GetNewBuffer();
    
    // Position is now invalid!
    var slice = buffer.Slice(position); // ❌ Undefined behavior
}

✅ Right:

public void ProcessMessages()
{
    var buffer = GetBuffer();
    
    // Use positions only within same sequence lifetime
    ProcessBuffer(buffer);
    
    // Get new buffer and new positions
    buffer = GetNewBuffer();
    SequencePosition newPosition = buffer.Start;
}

Mistake 4: Not Handling Empty Sequences

❌ Wrong:

public byte GetFirstByte(ReadOnlySequence<byte> sequence)
{
    return sequence.First.Span[0]; // Throws if empty!
}

✅ Right:

public bool TryGetFirstByte(ReadOnlySequence<byte> sequence, out byte value)
{
    value = default;
    
    if (sequence.IsEmpty)
    {
        return false;
    }
    
    value = sequence.First.Span[0];
    return true;
}

Mistake 5: Inefficient String Conversion

❌ Wrong:

public string ConvertToString(ReadOnlySequence<byte> sequence)
{
    // Creates temporary array!
    return Encoding.UTF8.GetString(sequence.ToArray());
}

✅ Right:

public string ConvertToString(ReadOnlySequence<byte> sequence)
{
    if (sequence.IsSingleSegment)
    {
        return Encoding.UTF8.GetString(sequence.First.Span);
    }
    
    // For multi-segment, renting is better than ToArray
    int length = (int)sequence.Length;
    byte[] rented = ArrayPool<byte>.Shared.Rent(length);
    
    try
    {
        sequence.CopyTo(rented);
        return Encoding.UTF8.GetString(rented, 0, length);
    }
    finally
    {
        ArrayPool<byte>.Shared.Return(rented);
    }
}

Performance Considerations 🚀

Benchmarking Single vs Multi-Segment

[MemoryDiagnoser]
public class SequenceBenchmarks
{
    private ReadOnlySequence<byte> _singleSegment;
    private ReadOnlySequence<byte> _multiSegment;
    
    [GlobalSetup]
    public void Setup()
    {
        var data = new byte[4096];
        _singleSegment = new ReadOnlySequence<byte>(data);
        
        // Create 8 segments of 512 bytes each
        var first = new BufferSegment(new byte[512]);
        var current = first;
        for (int i = 1; i < 8; i++)
        {
            current = current.Append(new byte[512]);
        }
        _multiSegment = new ReadOnlySequence<byte>(first, 0, current, 512);
    }
    
    [Benchmark]
    public int CountBytes_SingleSegment()
    {
        return (int)_singleSegment.Length;
    }
    
    [Benchmark]
    public int CountBytes_MultiSegment()
    {
        return (int)_multiSegment.Length;
    }
    
    [Benchmark]
    public int IterateSegments_Single()
    {
        int total = 0;
        foreach (var segment in _singleSegment)
        {
            total += segment.Length;
        }
        return total;
    }
    
    [Benchmark]
    public int IterateSegments_Multi()
    {
        int total = 0;
        foreach (var segment in _multiSegment)
        {
            total += segment.Length;
        }
        return total;
    }
}

Typical results:

Method	Mean	Allocated
CountBytes_SingleSegment	0.5 ns	0 B
CountBytes_MultiSegment	0.5 ns	0 B
IterateSegments_Single	12 ns	0 B
IterateSegments_Multi	85 ns	0 B

💡 Takeaway: Length property is always fast, but iteration cost scales with segment count. Always check IsSingleSegment when possible.

Integration with System.IO.Pipelines 🔄

ReadOnlySequence<T> is the heart of the Pipelines API:

public async Task ProcessPipelineAsync(PipeReader reader)
{
    while (true)
    {
        ReadResult result = await reader.ReadAsync();
        ReadOnlySequence<byte> buffer = result.Buffer;
        
        SequencePosition consumed = buffer.Start;
        SequencePosition examined = buffer.End;
        
        try
        {
            // Process complete messages in buffer
            while (TryParseMessage(buffer, out var message, out var position))
            {
                ProcessMessage(message);
                buffer = buffer.Slice(position);
                consumed = position;
            }
        }
        finally
        {
            // Tell pipeline how much we consumed
            reader.AdvanceTo(consumed, examined);
        }
        
        if (result.IsCompleted)
        {
            break;
        }
    }
}

Flow diagram:

┌─────────────────────────────────────────────┐
│         PipeReader Workflow                 │
└─────────────────────────────────────────────┘

    ┌──────────────┐
    │ ReadAsync()  │
    └──────┬───────┘
           │
           ▼
    ┌──────────────────────┐
    │ Get ReadOnlySequence │
    │ from ReadResult      │
    └──────┬───────────────┘
           │
           ▼
    ┌──────────────────────┐
    │ Parse messages       │
    │ Track positions      │
    └──────┬───────────────┘
           │
           ▼
    ┌──────────────────────┐
    │ AdvanceTo(consumed)  │
    │ (releases memory)    │
    └──────┬───────────────┘
           │
           └──────→ (repeat)

Key Takeaways 🎯

1. Zero-Copy Philosophy: ReadOnlySequence<T> enables processing discontiguous memory without copying—essential for high-performance scenarios.

2. Always Check IsSingleSegment: This simple check unlocks fast-path optimizations for contiguous data.

3. SequencePosition is Opaque: Treat positions as bookmarks—don't inspect internals, don't reuse across sequences.

4. Use SequenceReader: For complex parsing, SequenceReader<T> handles segment boundaries automatically.

5. Avoid ToArray(): This defeats the purpose. Use iteration or stackalloc for temporary copies instead.

6. Pipelines Integration: ReadOnlySequence<T> is designed to work seamlessly with System.IO.Pipelines for efficient I/O.

7. Memory Pooling: Combine with ArrayPool<T> and MemoryPool<T> for complete zero-allocation processing.

📚 Further Study

• Microsoft Docs: ReadOnlySequence
• System.IO.Pipelines: High performance I/O
• Memory and Span usage guidelines